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© Signal coding. 



© The current frame of the picture is compared (2) 
block-by-block with the previous frame to identify 
changed picture elements (pixels). The resulting ma- 
trix is matched f vector quantised 1 ) (4) to one of a set 
of standard matrices fVQ shapes'). Only those pix- 
els flagged by the selected VQ shape are transmit- 
ted plus a "VQ shape code". This technique is 
modified in that the current frame is also compared 
(30) with a "background" frame stored (31) at the 
transmitter and at the receiver. Where, for any block, 
all the pixels flagged by the VQ shape are indicated 
^ by the second comparison as being the same as the 
^background, the pixels are not sent - instead a 
reshape code plus a "copy background" instruction is 

©transmitted. 
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SIGNAL CODING 



The present invention relates to coding of vid- 
eo signals, especially using conditional replenish- 
ment coding, where information is generally trans- 
mitted only in respect of elements of a frame of the 
picture which have changed relative to a previous 
frame; the transmitted data being used at a re- 
ceiver to update a stored version of the picture. 

Thus picture elements of each block of an 
image to be coded are compared with those of the 
corresponding block of a previously coded image 
to determine whether the block has changed be- 
tween the two images; if so, picture element data 
are generated for output. 

Such a system is described in international 
patent application published under no. 
WO86/03922, which also proposes that the block 
be compared with the corresponding block of an 
earlier (reference) image. If they are deemed to be 
the same, no picture data are generated - instead, 
a codeword is produced to indicate that the re- 
ceiver is to obtain its data from a locally stored 
replica of the reference image. 

According to one aspect of the present inven- 
tion, there is provided a method of coding an 
image comprising, for each of a plurality of blocks 
of an image: 

(i) comparing picture elements of the block 
with those of the corresponding block of a pre- 
viously coded image to produce a matrix of values 
each indicating whether the corresponding element 
is, in accordance with a predetermined criterion, 
deemed to have changed between the two images; 

(ii) matching the matrix to one of a predeter- 
mined set of such matrices each of which identifies 
a region of the block as being deemed to have 
changed, and generating a codeword identifying 
that one matrix of the set; 

(iii) comparing picture elements of the block 
with those of the corresponding block of a refer- 
ence image composed of blocks from at least one 
earlier frame to determine whether the block is, in 
accordance with a predetermined criterion, deemed 
to have changed between the two images; 

(iv) coding for output those elements within 
the identified region, unless all the elements within 
the region are identified by comparison step (iii) as 
being unchanged relative to the reference image, 
whereupon a codeword indicating this is generated. 

In another aspect, the invention provides an 
apparatus for coding an image comprising, 

(i) means for comparing, for each of a plural- 
ity of blocks of an image, picture elements of the 
block with those of the corresponding block of a 



previously coded image to produce a matrix of 
values each indicating whether the corresponding 
element is, in accordance with a predetermined 
criterion, deemed to have changed between the 
s two images; 

(ii) means for matching the matrix to one of 
a predetermined set of such matrices each of 
which identifies a region of the block as being 
deemed to have changed, and generate a 

10 codeword identifying that one matrix of the set; 

(iii) a store arranged to store a reference 
image composed of blocks from at least one earlier 
frame and means for comparing picture elements 
of the block with those of the corresponding block 

75 of the reference image to determine whether the 
block is, in accordance with a predetermined cri- 
terion, deemed to have changed between the two 
images; 

(iv) means for coding for output those ele- 
20 ments within the identified region, unless ail the 

elements within the region are identified by com- 
parison step (iii) as being unchanged relative to the 
reference image, whereupon a codeword indicating 
this is generated. 

25 

The matching step - often termed vector quan- 
tisation - of two-dimensional maps has been pro- 
posed previously for picture coding, for example in 
European patent application serial no. 0239076A, 
30 where blocks of transform coefficient blocks are 
classified. 

The present invention, however, further exploits 
the vector quantisation, in that not only can the 
classification be used (if desired) to reduce the 
35 amount of information that has to be transmitted to 
indicate which elements have been coded for out- 
put, but also the number of occasions on which a 
"reference" codeword is instead generated are in- 
creased. 

40 Some embodiments of the invention will now 
be described, by way of example, with reference to 
the accompanying drawings, in which: 

- Figure 1 is a block diagram of a coder 
according to one embodiment of the invention; 

45 - Figure 2 shows a typical bit map produced 

by the change detector of the coder of Figure 1; 

- Figure 3 illustrates a few standard bit pat- 
terns used by the vector quantiser of the coder of 
Figure 1; and 

50 - Figure 4 is a block diagram of a decoder 

according to another embodiment of the present 
invention. 

Figure 1 shows a conditional replenishment 
video coder, where video signals (assumed to be in 
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digital form) are supplied to an input 1 . The current 
frame of a picture is compared in a change or 
movement detector 2 with the output of a locai 
decoder 3. The local decoder produces a "previous 
frame" output which is the same as that produced 

? by a remote decoder the object of the comparison 

is to identify those parts of the picture which have 
changed so that only information concerning those 

• parts needs to be sent to the decoder, to update a 

stored representation of the frame. 

The signals are processed on a block-by-block 
basis - an 8 x 8 block is assumed - and the change 
detector 2 therefore produces an 8 x 8 bit map. A 
typical map is shown in figure 2, where the ele- 
ments of the block corresponding to picture ele- 
ments (pixels) which have changed are shown 
shaded. 

In practice, transmission of information con- 
cerning only the changed elements involves a sig- 
nificant addressing overhead and therefore it is 
preferred to match the bit map to one of a limited 
number - typically forty - of standard shapes (a few 
are illustrated in figure 3). Since it is preferable to 
transmit information for an unchanged pixel than to 
fail to transmit information for a changed pixel, the 
shape chosen is the smallest (i.e. with the least 
number of shaded elements) which has a shaded 
area covering the shaded elements of the bit map. 
This process is termed vector quantisation (VQ) 
and is indicated as vector quantizer 4 in figure 1 . 
One transmits, for the block, a VQ number Identify- 
ing the chosen shape, along with information con- 
cerning pixels deemed - in accordance with the 
chosen shape - to have changed. The output data 
are combined and buffered in a buffer 5 prior to 
passage to an output 6. As is conventional in such 
systems, the buffer is used to smooth variations in 
the rate at which data are generated (due to the 
picture content dependent coding) and interface to 
a regular transmitted rate and the buffer fullness 
state used to control the rate of generation (e.g. by 
varying the change detector thresholds (control line 
7)). 

The pixels deemed to have changed are coded 
in an encoder 8 (for example in the manner de- 
scribed in our International patent application 
PCT/GB88/00709 (publication no. ) and Eu- 
ropean patent application no. 88307981.6 
(publication no. ). 

Many televised scenes - especially in a 
videoconference or videotelephone environment - 
contain moving persons or objects set against a 
fixed background. Pixels observed as changed by 
the change detector 2 will relate either to objects 
which have changed their position (or entered the 
scene) or to parts of the background uncovered by 
the object. In the present coder, a second change 
detector 30 is also shown, which compares the 



current frame with a reference or background 
frame stored in a frame store 31. Acquisition of the 
stored background frame will be discussed further 
below, but in figure 1 is assumed to have been 
5 acquired from the local decoder 3 in response to 
manual operation of a switch 32 at the commence- 
ment of a transmission; a code being transmitted to 
the receiver to initiate similar action at the remote 
decoder. 

10 The change detector 30 produces a bit map 
identifying those pixels of the current frame which 
are the same as the background. 

If this shows that the new image is different 
from the background for any of the pixels declared 

75 as changed by the VQ shape the background in- 
formation cannot be used and information concern- 
ing those pixels is transmitted along with the VQ 
number. 

If, however the changed area as given by the 

20 VQ shape covers only pixels which are identified 
by the detector 30 as being the same as the 
corresponding pixels of the stored background 
frame, then the VQ number is accompanied by a 
reserved codeword indicating "background" and no 

25 further information needs to be transmitted for the 
block in question. 

These functions are accomplished in figure 2 
by a comparator 33 whose output 34 overrides the 
operation of the encoder 8. 

30 Figure 4 shows a decoder. An input processor 
40 receives the coded input signals. The back- 
ground frame is stored in a background store 41 . In 
normal operation, the processor 40 uses the pixel 
information received to update via line 42 the con- 

35 tents of a frame store 43, using the received VQ 
numbers via line 44 to control the frame store 
addressing. When,' however, it receives the re- 
served "background" codeword (via line 45), it 
recovers the relevant pixels (identified by the VQ 

40 number) from the background store 41 and enters 
then into the frame store 43. This is illustrated 
schematically by a changeover switch 46. The 
frame store 43 is read out (by output control means 
47) to produce the received video at an output 48. 

45 As in the coder, the background store 41 is loaded 
from the decoded image in store 43 when a 'load 
background 1 instruction is received (line 49). 

Note that the local decoder 3 of Figure 1 can 
be of the same construction as the decoder of 

so Figure 4, although, of course, in practice the locai 
decoder would use the background store 31 rather 
than contain one of its own. 

The change detectors 2,30 can in principle be 
any conventionally used, or may both be as de- 

55 scribed in our above-mentioned patent applications. 
The vector quantiser 4 can again be a known 
device. One possibility is that described in our 
International patent application no. 
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PCT/GB87/00816 {publication no. WO88/04084) 
and European patent application no. 8627787 
(publication no. 0272794). 

Simpler but cruder, another option is to convert 
the 8x8 matrix to a 4x4 matrix by creating each 
'new' element as an OR function of four 'old 1 ele- 
ments; this reduces the number of elements in the 
matrix to a size (16) which can be used to address 
a look-up table - in the form of a 64. Kbyte read 
only memory in which the appropriate VQ numbers 
are stored. 

The background scene may be 'frozen' man- 
ually at the commencement of a transmission. It 
may, but does not have to, consist solely of a fixed 
background. For example it may include seated 
figures (thereby covering the situation where a per- 
son momentarily passes his hand over his face if 
the face forms part of the "background", the face 
does not have to be retransmitted). 

It may be desirable to include provision for 
updating the background. For example, the incom- 
ing video could be monitored at the coder (by a 
unit 35 shown dotted in Figure 1) and parts of the 
picture which differ from the original background 
but have remained unchanged for a predetermined 
period of time inserted into the frame store, a 
signal being sent (via line 36) to the decoder to 
instruct it to do likewise. 

An alternative method of updating the back- 
ground store abandons any attempt at identifying 
genuine background, but instead forms a reference 
image which is a composite of blocks taken from 
preceding images over a period. Each frame pe- 
riod, data for a few (e.g. eight) selected blocks 
scattered over the image area is entered into the 
background stores, the block selection being such 
that different blocks are selected from each frame 
until the whole image area has been covered. As- 
suming 1024 blocks per frame at 25 frames per 
second, this represents a period of approximately 
40 seconds. The biocks could be loaded from the 
frame store 43 of the decoder, and a correspond- 
ing store in the local decoder 3, the change moni- 
tor then being replaced by a simple address gener- 
ator 35 to select the appropriate blocks. Clearly, 
this is less effective, in that some blocks of the 
reference image will not represent background ma- 
terial, but a significant coding advantage is still 
obtained, and It has the merit of simplicity. 

A variation of this approach provides that data 
for selected blocks, instead of being drawn from 
the decoded image, are actually transmitted - i.e. 
transmission of the whole block is forced even if 
only a part, or none, of it is indicated by the 
detector 2 and quantiser 4 as moving. In this case, 
the frame stores 31, 41 take their input from the 
input 1 and processor 40 respectively. This also 



has the benefit of ensuring that transmission errors 
on blocks which rarely change do not persist in the 
decoded image. 

5 

Claims 

1. A method of coding an image comprising, 
for each of a plurality of blocks of an image: 

10 (i) comparing picture elements of the block 

with those of the corresponding block of a pre- 
viously coded image to produce a matrix of values 
each indicating whether the corresponding element 
is, in accordance with a predetermined criterion, 

75 deemed to have changed between the two images; 

(II) matching the matrix to one of a predeter- 
mined set of such matrices each of which identifies 
a region of the block as being deemed to have 
changed, and generating a codeword identifying 

20 that one matrix of the set; 

(iii) comparing picture elements of the block 
with those of the corresponding block of a refer- 
ence image composed of blocks from at least one 
earlier frame to determine whether the block is, in 

25 accordance with a predetermined criterion, deemed 
to have changed between the two images; 

(iv) coding for output those elements within 
the identified region, unless all the elements within 
the region are identified by comparison step (iii) as 

30 being unchanged relative to the reference image, 
whereupon a codeword indicating this is generated. 

2. A method according to claim 1 in which, in 
the coding of the elements within the identified 

35 region, at least some of the elements are coded as 
the difference between that element and a pre- 
dicted value for that element derived from one or 
more previously coded elements of the block, the 
sequence of coding of the elements being depen- 

40 dent on the orientation of the identified region. 

3. An apparatus for coding an image compris- 
ing, 

(i) means for comparing, for each of a plural- 
ity of blocks of an image, picture elements of the 

45 block with those of the corresponding block of a 
previously coded Image to produce a matrix of 
values each indicating whether the corresponding 
element is, in accordance with a predetermined 
criterion, deemed to have changed between the 

50 two images; 

(ii) means for matching the matrix to one of 
a predetermined set of such matrices each of 
which identifies a region of the block as being 
deemed to have changed, and generate a 

55 codeword identifying that one matrix of the set; 

(iii) a store arranged to store a reference 
image composed of blocks from at least one earlier 
frame and means for comparing picture elements 
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of the block with those of the corresponding block 
of the reference image to determine whether the 
block is, in accordance with a predetermined cri- 
terion, deemed to have changed between the two 
images; 5 

(iv) means for coding for output those ele- 
ments within the identified region, unless all the 
elements within the region are identified by com- 
parison step (iii) as being unchanged relative to the 
reference image, whereupon a codeword indicating io 
this is generated. 

4. An apparatus according to claim 3, including 
means for comparing successive images to identify 
parts of the image which have remained un- 75 
changed for a predetermined period of time, and in 

the event of such parts being identified, to: 

(a) update the stored reference image; and 

(b) code for output data indicating which parts of 

the reference image have been thus updated. 20 

5. An apparatus according to claim 3, including 
means arranged in operation to effect periodic re- 
placement of a minority of the blocks of the refer- 
ence image stored in the store by the correspond- 
ing blocks of a recent image, the blocks being 25 
differently selected for successive replacements 
such that the store always contains a composite 
image made up of blocks from a plurality of frames 

of the image being coded. 

6. A decoder for use with the coder of claim 3 30 
or 5, comprising: 

(a) a frame store for storing a received image; 

(b) a second frame store; 

(c) control means responsive to received data to 
update the contents of the second frame store 35 
using the received data and responsive to a re- 
ceived codeword to update the second frame store 

with information from the first frame store; 

(d) means for repetitively reading the contents of 

the second frame store to produce a video output 40 
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